78 research outputs found
EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantization
Mixed-Precision Quantization~(MQ) can achieve a competitive
accuracy-complexity trade-off for models. Conventional training-based search
methods require time-consuming candidate training to search optimized per-layer
bit-width configurations in MQ. Recently, some training-free approaches have
presented various MQ proxies and significantly improve search efficiency.
However, the correlation between these proxies and quantization accuracy is
poorly understood. To address the gap, we first build the MQ-Bench-101, which
involves different bit configurations and quantization results. Then, we
observe that the existing training-free proxies perform weak correlations on
the MQ-Bench-101. To efficiently seek superior proxies, we develop an automatic
search of proxies framework for MQ via evolving algorithms. In particular, we
devise an elaborate search space involving the existing proxies and perform an
evolution search to discover the best correlated MQ proxy. We proposed a
diversity-prompting selection strategy and compatibility screening protocol to
avoid premature convergence and improve search efficiency. In this way, our
Evolving proxies for Mixed-precision Quantization~(EMQ) framework allows the
auto-generation of proxies without heavy tuning and expert knowledge. Extensive
experiments on ImageNet with various ResNet and MobileNet families demonstrate
that our EMQ obtains superior performance than state-of-the-art mixed-precision
methods at a significantly reduced cost. The code will be released.Comment: Accepted by ICCV202
GROVE: A Retrieval-augmented Complex Story Generation Framework with A Forest of Evidence
Conditional story generation is significant in human-machine interaction,
particularly in producing stories with complex plots. While Large language
models (LLMs) perform well on multiple NLP tasks, including story generation,
it is challenging to generate stories with both complex and creative plots.
Existing methods often rely on detailed prompts to guide LLMs to meet target
conditions, which inadvertently restrict the creative potential of the
generated stories. We argue that leveraging information from exemplary
human-written stories facilitates generating more diverse plotlines. Delving
deeper into story details helps build complex and credible plots. In this
paper, we propose a retrieval-au\textbf{G}mented sto\textbf{R}y generation
framework with a f\textbf{O}rest of e\textbf{V}id\textbf{E}nce (GROVE) to
enhance stories' complexity. We build a retrieval repository for target
conditions to produce few-shot examples to prompt LLMs. Additionally, we design
an ``asking-why'' prompting scheme that extracts a forest of evidence,
providing compensation for the ambiguities that may occur in the generated
story. This iterative process uncovers underlying story backgrounds. Finally,
we select the most fitting chains of evidence from the evidence forest and
integrate them into the generated story, thereby enhancing the narrative's
complexity and credibility. Experimental results and numerous examples verify
the effectiveness of our method.Comment: Findings of EMNLP 202
A Case-Based Reasoning Framework for Adaptive Prompting in Cross-Domain Text-to-SQL
Recent advancements in Large Language Models (LLMs), such as Codex, ChatGPT
and GPT-4 have significantly impacted the AI community, including Text-to-SQL
tasks. Some evaluations and analyses on LLMs show their potential to generate
SQL queries but they point out poorly designed prompts (e.g. simplistic
construction or random sampling) limit LLMs' performance and may cause
unnecessary or irrelevant outputs. To address these issues, we propose
CBR-ApSQL, a Case-Based Reasoning (CBR)-based framework combined with GPT-3.5
for precise control over case-relevant and case-irrelevant knowledge in
Text-to-SQL tasks. We design adaptive prompts for flexibly adjusting inputs for
GPT-3.5, which involves (1) adaptively retrieving cases according to the
question intention by de-semantizing the input question, and (2) an adaptive
fallback mechanism to ensure the informativeness of the prompt, as well as the
relevance between cases and the prompt. In the de-semanticization phase, we
designed Semantic Domain Relevance Evaluator(SDRE), combined with Poincar\'e
detector(mining implicit semantics in hyperbolic space), TextAlign(discovering
explicit matches), and Positector (part-of-speech detector). SDRE semantically
and syntactically generates in-context exemplar annotations for the new case.
On the three cross-domain datasets, our framework outperforms the
state-of-the-art(SOTA) model in execution accuracy by 3.7\%, 2.5\%, and 8.2\%,
respectively
Retrieval-augmented GPT-3.5-based Text-to-SQL Framework with Sample-aware Prompting and Dynamic Revision Chain
Text-to-SQL aims at generating SQL queries for the given natural language
questions and thus helping users to query databases. Prompt learning with large
language models (LLMs) has emerged as a recent approach, which designs prompts
to lead LLMs to understand the input question and generate the corresponding
SQL. However, it faces challenges with strict SQL syntax requirements. Existing
work prompts the LLMs with a list of demonstration examples (i.e. question-SQL
pairs) to generate SQL, but the fixed prompts can hardly handle the scenario
where the semantic gap between the retrieved demonstration and the input
question is large. In this paper, we propose a retrieval-augmented prompting
method for a LLM-based Text-to-SQL framework, involving sample-aware prompting
and a dynamic revision chain. Our approach incorporates sample-aware
demonstrations, which include the composition of SQL operators and fine-grained
information related to the given question. To retrieve questions sharing
similar intents with input questions, we propose two strategies for assisting
retrieval. Firstly, we leverage LLMs to simplify the original questions,
unifying the syntax and thereby clarifying the users' intentions. To generate
executable and accurate SQLs without human intervention, we design a dynamic
revision chain which iteratively adapts fine-grained feedback from the
previously generated SQL. Experimental results on three Text-to-SQL benchmarks
demonstrate the superiority of our method over strong baseline models
Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models
Most open-domain dialogue systems suffer from forgetting important
information, especially in a long-term conversation. Existing works usually
train the specific retriever or summarizer to obtain key information from the
past, which is time-consuming and highly depends on the quality of labeled
data. To alleviate this problem, we propose to recursively generate summaries/
memory using large language models (LLMs) to enhance long-term memory ability.
Specifically, our method first stimulates LLMs to memorize small dialogue
contexts and then recursively produce new memory using previous memory and
following contexts. Finally, the LLM can easily generate a highly consistent
response with the help of the latest memory. We evaluate our method using
ChatGPT and text-davinci-003, and the experiments on the widely-used public
dataset show that our method can generate more consistent responses in a
long-context conversation. Notably, our method is a potential solution to
enable the LLM to model the extremely long context. Code and scripts will be
released later
Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Tasks
Text classification tasks often encounter few shot scenarios with limited
labeled data, and addressing data scarcity is crucial. Data augmentation with
mixup has shown to be effective on various text classification tasks. However,
most of the mixup methods do not consider the varying degree of learning
difficulty in different stages of training and generate new samples with one
hot labels, resulting in the model over confidence. In this paper, we propose a
self evolution learning (SE) based mixup approach for data augmentation in text
classification, which can generate more adaptive and model friendly pesudo
samples for the model training. SE focuses on the variation of the model's
learning ability. To alleviate the model confidence, we introduce a novel
instance specific label smoothing approach, which linearly interpolates the
model's output and one hot labels of the original samples to generate new soft
for label mixing up. Through experimental analysis, in addition to improving
classification accuracy, we demonstrate that SE also enhances the model's
generalize ability
The epithelial transcriptome and mucosal microbiota are altered for goats fed with a low-protein diet
IntroductionFeeding low protein (LP) diet to animals impose severe challenge to animals' immune homeostasis. However, limited knowledge about the underlying adaption mechanism of host and ruminal microbiota responding to LP diet were well understood. Herein, this study was performed to examine the changes in relative abundance of ruminal microbiota and host ruminal mucosal transcriptome profiles in response to a LP diet.MethodsA total of twenty-four female Xiangdong balck goats with similar weight (20.64 ± 2.40 kg) and age (8 ± 0.3 months) were randomly assigned into two groups, LP (5.52% crude protein containing diet) and CON (10.77% crude protein containing diet) groups. Upon completion of the trial, all goats were slaughtered after a 16-hour fasting period in LiuYang city (N 28°15′, E 113°63′) in China. HE staining, free amino acids measurement, transcriptome analysis and microbiome analysis were applied to detect the morphology alterations, free amino acids profile alterations and the shift in host ruminal mucosal transcriptome and ruminal microbiota communities.ResultsFirstly, the results showed that feeding LP diet to goats decreased the rumen papilla width (P = 0.043), surface area (P = 0.013) and total ruminal free amino acids concentration (P = 0.016). Secondly, microbiome analysis indicated that 9 microbial genera, including Eubacterium and Prevotella, were enriched in LP group while 11 microbial genera, including Butyrivibrio and Ruminococcus, were enriched in CON group. Finally, in terms of immune-related genes, the expression levels of genes involved in tight junction categories (e.g., MYH11, PPP2R2C, and MYL9) and acquired immunity (e.g., PCP4 and CXCL13) were observed to be upregulated in the LP group when compared to the CON group.ConclusionUnder the LP diet, the rumen exhibited increased relative abundance of pathogenic microbiota and VFA-degrading microbiota, leading to disruptions in immune homeostasis within the host's ruminal mucosa. These findings indicate that the ruminal microbiota interacts with host results in the disruption in animals' immune homeostasis under LP diet challenge
- …